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Abstract Positive bases, which play a key role in understanding derivative free op¬ 
timization methods that use a direct search framework, are positive spanning sets 
that are positively linearly independent. The cardinality of a positive basis in IR" has 
been established to be between n+1 and 2n (with both extremes existing). The lower 
bound is immediate from being a positive spanning set, while the upper bound uses 
both positive spanning and positively linearly independent. In this note, we provide 
details proving that a positively linearly independent set in IR" for n G {1,2} has at 
most 2n elements, but a positively linearly independent set in IR" for n > 3 can have 
an arbitrary number of elements. 
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1 Introduction 


First introduced in 1954 IIDav54l . the notion of a positive basis plays a key role in un¬ 
derstanding derivative free optimization methods that use a direct search framework 
(a.k.a. pattern search methods) IILT96IICP02IIKLT0^ Reg 1 5| . In loose terms, a pos¬ 
itive basis is a set that provides some directional information into every half-space, 
without including unnecessary vectors. 

Pattern search methods seek a minimizer of / by evaluating / over an ever shrink¬ 
ing ‘pattern’; 


Given / and a incumbent solution x^. 

- Evaluate /(x^ + au*) for each v’’ G V = ..., u'"}. 
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- If a lower function value is found, then update 

- If no lower function value is found, then shrink a. 

If f € C^, V f{x^) ^ 0, and the pattern forms a positive basis, then at least one vector 
in the pattern is a descent direction for / at Thus, under these conditions, once a 
is sufficiently small, descent must occur. Formalization of this idea has been used to 
prove convergence in a number of direct search/pattern search methods IILT96IICP02I 
IKLT03IIAD04I (and many others). 

Mathematically, to define positive basis we begin with the positive span: given a 
set of vectors V = ..., w™} C IR", we define the positive span of V by 

f ™ . 'I 

p-span(V) = < ^ ctiU* : > 0 for all i = 1, 2,..., m > . 

We now define V to be a positive basis if it satisfies two properties 

i) [Positive Spanning] p-span(V) = H", and 

ii) [Positive Linear Independence] u* ^ p-span(V\{u*}) for all i = 1,2,..., m. 

Unlike bases of IR" (which always have exactly n elements), a positive basis of H" 
has between n + 1 and 2n elements. The lower bound of n + 1 comes from the 
following fact. 

Proposition! HDav54\ Thm3.7]IfV = ..., u™} is a positive spanning set 

o/IR", then V® = ..., is a spanning set o/IR". 

Thus, to satisfy positive spanning condition, a set must contain at least n + 1 
vectors. 

The upper bound on the cardinality of a positive basis is more subtle to obtain. The 
original work of Davis includes a proof of this fact 0Dav54l Thm 6.7], but the deriva¬ 
tion is rather technical. Alternate proofs appear in the works of Shephard IIShe71l and 
Audet OAudl II . 

It is important to note that all three proofs use both positive spanning and positive 
linear independence to provide the upper bound. As such, the property of positive 
linear independence is less clear in its implications on cardinality. For example, any 
set consisting of exactly one vector is always positive linearly independent - includ¬ 
ing the set V = {0}. (Of course, any positive linearly independent set with 2 or more 
vectors cannot contain the 0 vector.) Based on a knowledge of positive bases, one 
might conjecture that positively linearly independent sets have at most 2n elements. 
This is true in IR^ and but false beyond that. In this note, we provide details 
proving that a positively linearly independent set in IR" for n G {1,2} has at most 
2n elements, but a positively linearly independent set in H" for n > 3 can exceed 
this bound. Proofs of the first two statements appear in Section |2 In Section |3] we 
provide two examples. The first is a straightforward example of a positively linearly 
independent set V with 



This example requires no tools other than basic linear algebra. The second example 
develops a positively linearly independent set in IR" with an arbitrary number of 
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elements. This example requires a small knowledge of convex analysis, but is still 
quite accessible to a general audience. 

We conclude the introduction with a lemma that allows us to simplify notation in 
all future proofs and examples. 

Lemma 1 Let V = ..., u"*} C H” with u® ^ 0/or all i. Then V is pos¬ 
itively linearly independent if and only ifV = IhHl’''' ’ ||"™|| I positively 

linearly independent. 


Proof. Note that 


aiU* 

i^k 


-^ = V/3— 


where cti > 0 and fii = jp7|- > 0 for any i f k. Thus, V is positively linearly 

independent if and only if V is positively linearly independent. □ 


2 Dimensions IR^ and IR^ 

Proposition 2 The maximal cardinality of a positively linearly independent set in 
is 2. 

Proof. Suppose V = If 0 S V, then V is not positively linearly inde¬ 

pendent. If 0 ^ V, then, by Lemma [1] without loss of generality u* G {—1,1} for 
each i. By the pigeon hole principle, at least two vectors i and j are the same, and V 
is not positively linearly independent. Thus, if V is positively linearly independent, 
then |V| <2. □ 

Proposition 3 The maximal cardinality of positively linearly independent sets in IR^ 
is 4. 

Proof. For eventual contradiction, suppose V = ..., u®} C ]R^ is positively 

linearly independent. Without loss of generality, assume ||t;®|| = 1 for z = 1, 2,..., 5. 
Clearly, f u® for any i f \. Moreover, if = —u* for some i, then f —v^ 
for any k f i. As such, there are at least two vectors, say and that are linearly 
independent. Examining, V \ and repeating the same arguments, we may 

assume that and are also linearly independent. Therefore, {v^,v^} and {v^, 
are both bases of IR^. 

As is a basis of IR^, there exists a G IR^ such that = aiv^ + a 2 V^. 

If ai > 0 and a 2 > 0, then we have contradicted the positive linear independence of 
V. If cti >0 and a 2 < 0, then aiv^ = — a 2 V^, which also contradicts positive 

linear independence. Similarly, a 2 > 0 and ai < 0 fails. Hence, a < 0. 

Using the same arguments, and is a basis of IR^, there exists /3 < 0 and 

7 < 0 such that v'^ = Piv^ + and = 71 + 72 ^^. Substitution now yields 

= (71 ai + 72 / 3 i)u^ + (71 ^2 + 72/32 
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which shows that S p-span(V\{t;®}), so V is not positively linearly independent. 

□ 

On a final note for this section, it is clear that the maximum cardinalities derived 
above can be achieved, as constructing an example with | V| = 2n is trivial. 


3 Dimension IR” for n > 3 


Our first example provides a family of positively linear independent sets whose car¬ 
dinality grows exponentially with dimension. 


Example 1 In IR”, let V be the set of vectors of all the nontrivial permutations with 
[fj I’sandn- [fj = [f] O’s, 


V = 



{o,ir 


n 





then |V| = and V is positively linearly independent. 

Proof. Suppose € V and there exist nonnegative numbers ai such that = 
As f 0, there must exist some j with > 0. Since f vf and 
€ V, there must exist some index I with vf = 0 and vf = 1. This yields the 
contridiction 

0 = Vi = ^ aivl > ajvf = aj > 0 . 
i^k 

Hence, V is positively linearly independent. The proof of the cardinality is trivial. □ 
Our second example requires some basic tools and definitions from convex anal¬ 
ysis. 


Definition 1 A set O C IR" is convex, if given any x,y € C and any 9 G [0,1], the 
point z = 9x + (1 — 9)y G C. 

A set C C IR" is strictly convex, if given any x,y G C with x f y and any 9 G (0,1), 
the point z = 9x + {1 — 9)y G int(C'). 

Definition 2 The convex hull of a set S, denoted conv(S'), is the smallest convex set 
that contains S. 


Example 2 The set Bi = {x G IR" : ||a;|| < 1} is strictly convex. 
Example 3 IIRW091 Thm 2.27] Let Y = {y^, y^, ..., y"*} C C. Then 


conv(F) 



Combining definition [T] and example[3 we have the following classic lemma. 
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Lemma 2 Suppose C is strictly convex. Let Y = ...y"*} C C and x G 

conv(y). Then 

either X = for some i or x G int(C'). 

We now prove the main result. 

Theorem 1 For n > Z a positively linearly independent set in IR" may contain an 
arbitrary number of vectors. 

Proof. We shall provide an example in Extending to H" can be trivially accom¬ 
plished by adding 0 elements in the extra dimensions. 

Let S = {{x^y) : x^ + = 1}. Select any distinct m points in S, 

and define 

V = {v\v\ ...,v^} = {{x\y\ 1), {x\y\ 1),..., 1)}. 

We claim that V is positively linear independent. 

Suppose there exists G p-span(R\{v^}). By reordering, without loss of gen¬ 
erality, assume k = m,so 


m—1 

u™ = ^ with ai > 0. 

i=l 

Noting that = 1 for all i, we must have 

m—1 771—1 

1 = = X! = X] 

1=1 1=1 


This implies 


m — 1 


m—1 




Oii{x\y^), ^ ai = 1, 


> 0 . 


I.e., G couv{{{x^,y^),{x^,y^),... ,{x"^~^,y‘^~^)}). As 

(x®,y*) for any i < m, Fact|2]implies (x’",y’") G int(i?i), contradicting (x*,y*) G 

i3i \ int(i3i) for all*. □ 

It is interesting to note that the example in the proof of Theorem [T] does not 
actually require m to be finite. Indeed, in it is possible to create an uncountable set 
of vectors V such that v ^ p-span(V\{t;}) for all v G V. Moreover, while the example 
uses the unit sphere to create the set, it is clear that S = {(x,y) : x^ -f = 1} could 
be replaced by the boundary of any compact strictly convex set. 

Finally, we note that an alternate (but similar) proof of Theorem [T] was indepen¬ 
dently developed by Regis and presented in [Regl5| Thm 3.4]. 
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